How much is enough? Proposing achievement thresholds for core EPAs of graduating medical students in Canada.

PURPOSE: The transition towards Competency-Based Medical Education at the Cumming School of Medicine was accelerated by the reduced clinical time caused by the COVID-19 pandemic. The purpose of this study was to define a standard protocol for setting Entrustable Professional Activity (EPA) achievement thresholds and examine their feasibility within the clinical clerkship. METHODS: Achievement thresholds for each of the 12 AFMC EPAs for graduating Canadian medical students were set by using sequential rounds of revision by three consecutive groups of stakeholders and evaluation experts. Structured communication was guided by a modified Delphi technique. The feasibility/consequence models of these EPAs were then assessed by tracking their completion by the graduating class of 2021. RESULTS: The threshold-setting process resulted in set EPA achievement levels ranging from 1 to 8 across the 12 AFMC EPAs. Estimates were stable after the first round for 9 of 12 EPAs. 96.27% of EPAs were successfully completed by clerkship students despite the shortened clinical period. Feasibility was predicted by the slowing rate of EPA accumulation overtime during the clerkship. CONCLUSION: The process described led to consensus on EPA achievement thresholds. Successful completion of the assigned thresholds was feasible within the shortened clerkship.[Box: see text].

Development of self-regulated lifelong learning (SR3Ls) model in the era of IR4.0 for post-pandemic economy

Purpose: To continue to stay relevant in the era of Industry Revolution 4.0 (IR4.0) alongside the unprecedented disruption of COVID-19, the importance of lifelong learning is indisputable though this concept has existed for decades. In this context, open and distance learning (ODL) institutions are urged to re-think and re-design their online learning support systems that inculcate self-regulated lifelong learning (SR3Ls) in their learners to be adaptable and resilient for the post-pandemic economy. The purpose of this paper is to develop a SR3Ls model, namely SR3Ls model by utilising the collective opinions of a panel of experts to determine the key domains and attributes. Design/methodology/approach: A 2-round Delphi consensus study was conducted with 39 experts from five countries. The mean, standard deviation (SD), inter-quartile range (IQR) and the ratio of experts assigned score of 4 or greater were used as the basis of consensus assessment with criteria set at mean = 3.0, SD = 1.5, IQR = 1, ratio on score 4 or greater at = 75%. The questionnaire consisted of 5-point Likert-type scale rating the importance level of each attribute combined with open-ended questions. Findings: This paper presented the findings of the first round of Delphi consensus study. For the first round, the experts were asked to evaluate 31 key attributes of SR3Ls model under five domains. The findings revealed that there were five key attributes to be eliminated from the list, while there were seven attributes identified as the key attributes with highest consensus. There were additional attributes suggested by the Delphi panel to be added in the second round of evaluation. Originality/value: This international consensus-based SR3Ls model serves as an important benchmark for ODL institutions across the regions in developing meaningful and relevant online learning support systems for their learners to adopt SR3Ls attributes in order to meet the dynamic market demands. © 2023, Fei Ping Por and Balakrishnan Muniandy.

Development of Reference Process Model and Reference Architecture for Pharmaceutical Cold Chain

Although the competence level in pharmaceutical cold chain logistics is adequate in the current healthcare sector, the future will be more unpredictable, disruptive, and chaotic than the world of today because of rapid technological changes, as well as social disruption. This work introduces and evaluates a new cold chain structure based on the enhanced reference process model (RPM) and reference architecture (RA) for the pharmaceutical cold chain competence design. The modified Delphi technique is used to design a systematic decision context to evaluate the improved RPM and RA and produce a competence design by obtaining agreement from a panel of experts. Ten experts in cold chain logistics took part in the modified Delphi assessment process to describe the model and architecture for an evaluation lead. They defined it as the assessment associated with the professional related to cold chain accreditation. Ten participants from six countries were asked questions regarding their expertise, abilities, and opinions in the first round, and their comments were collected. In the second and third rounds, comments and consensus were collected, which were set at 80% for RPM and RA. This work proposes an innovative scheme for developing occupational standards based on the RPM and RA, as opposed to the traditional method of employing functional analysis to establish occupational standards. This work can be further expanded to develop professional competencies in the pharmaceutical cold chain. © 2023 by the authors.

Engaging Stakeholders to Improve Care for Adults with Complex Health and Social Needs during a Public Health Emergency

A public health emergency such as the COVID-19 pandemic exacerbates the already challenging environment facing adults with complex health and social needs (ACHSN) and the systems of care that support them. Between September 2020 and April 2021, 51 participants representing six different stakeholder groups were engaged using interviews, asynchronous Delphi surveys, and a virtual stakeholder meeting to learn from their perspectives about the greatest needs and possible solutions affecting ACHSN populations during the COVID-19 pandemic and to develop a prioritized research agenda to improve care for ACHSN populations. Mental health and financial concerns were strongly and consistently endorsed as the most important issues. Future research priorities identified included both macro systems research such as testing alternative state-level models of payment for physical and mental health care and research that could be conducted at a local level (such as identifying needs for patient care navigation services and testing models of care navigation).

What Makes a Quality Health App-Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study.

BACKGROUND: The lack of an international standard for assessing and communicating health app quality and the lack of consensus about what makes a high-quality health app negatively affect the uptake of such apps. At the request of the European Commission, the international Standard Development Organizations (SDOs), European Committee for Standardization, International Organization for Standardization, and International Electrotechnical Commission have joined forces to develop a technical specification (TS) for assessing the quality and reliability of health and wellness apps. OBJECTIVE: This study aimed to create a useful, globally applicable, trustworthy, and usable framework to assess health app quality. METHODS: A 2-round Delphi technique with 83 experts from 6 continents (predominantly Europe) participating in one (n=42, 51%) or both (n=41, 49%) rounds was used to achieve consensus on a framework for assessing health app quality. Aims included identifying the maximum 100 requirement questions for the uptake of apps that do or do not qualify as medical devices. The draft assessment framework was built on 26 existing frameworks, the principles of stringent legislation, and input from 20 core experts. A follow-up survey with 28 respondents informed a scoring mechanism for the questions. After subsequent alignment with related standards, the quality assessment framework was tested and fine-tuned with manufacturers of 11 COVID-19 symptom apps. National mirror committees from the 52 countries that participated in the SDO technical committees were invited to comment on 4 working drafts and subsequently vote on the TS. RESULTS: The final quality assessment framework includes 81 questions, 67 (83%) of which impact the scores of 4 overarching quality aspects. After testing with people with low health literacy, these aspects were phrased as "Healthy and safe," "Easy to use," "Secure data," and "Robust build." The scoring mechanism enables communication of the quality assessment results in a health app quality score and label, alongside a detailed report. Unstructured interviews with stakeholders revealed that evidence and third-party assessment are needed for health app uptake. The manufacturers considered the time needed to complete the assessment and gather evidence (2-4 days) acceptable. Publication of CEN-ISO/TS 82304-2:2021 Health software - Part 2: Health and wellness apps - Quality and reliability was approved in May 2021 in a nearly unanimous vote by 34 national SDOs, including 6 of the 10 most populous countries worldwide. CONCLUSIONS: A useful and usable international standard for health app quality assessment was developed. Its quality, approval rate, and early use provide proof of its potential to become the trusted, commonly used global framework. The framework will help manufacturers enhance and efficiently demonstrate the quality of health apps, consumers, and health care professionals to make informed decisions on health apps. It will also help insurers to make reimbursement decisions on health apps.

Predictions for architectural and urban spaces in response to the COVID-19 pandemic: Delphi survey of Korean experts

Studies on the factors of the COVID-19 pandemic that influence architecture and spaces have presented various, often contradictory, findings, and the same is true for studies making predictions. Considering this, this study aims to use the Delphi technique, an analytical method for synthesizing the opinions of experts across diverse fields to determine major issues in the COVID-19 and post-COVID-19 eras as wells as the architectural and urban spaces in which future changes are expected. This study derived keywords representing major trends and issues that would lead to changes in architectural and urban spaces in the COVID-19 and post-COVID-19 eras, predicted the change patterns for each keyword, and determined the architectural and urban spaces expected to undergo major changes. The experts predicted these keywords to show a variety of changes, including the pattern of increasing influence during the COVID-19 pandemic and then decreasing in influence after the pandemic, the pattern of small influence during the pandemic and the increase in influence after the pandemic, and the pattern of greater influence during and after the pandemic. Furthermore, they predicted that most of the post-COVID-19 changes would occur in the housing sector. Developing architectural guidelines that could incorporate these changes is thus necessary. © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group on behalf of the Architectural Institute of Japan, Architectural Institute of Korea and Architectural Society of China.

The underdog effect: Towards a conceptual framework for enhancing voluntourism

Voluntourism is a unique tourism pattern which got considerable attention recently, especially after the COVID-19 pandemic and the intense search to promote and revive all the possible tourist patterns in order to support the tourism industry. This paper studied and analyzed voluntourism from a business perspective by developing Business Model Canvas (BMC) to support this tourism pattern. In order to develop Business Model Canvas for Voluntourism (BMC4V), this study relied on a Delphi Technique. A total of 22 experts, academics, stakeholders and officials active in voluntourism were invited to four-round Delphi workshops to identify the main components of the voluntourism industry. The developed BMC4V is structured from nine building blocks: key partners, key activities, key resources, value propositions, customer relationships, customer segments, channels, cost structure and revenue streams. The main contribution of this study is presenting BMC4V and guidelines for the stakeholders in ethically supporting the voluntourism industry. The results revealed that voluntourism has great potential to be a promising tourism pattern if we consider it a business model and not only a leisure tourism activity. Additionally, this study presented a key guideline for the main five actors in the voluntourism industry (voluntourists, sending organizations, servicing organizations, hosting organizations and government) to set this tourism pattern on the right track. Management implications Most of the scientific literature that discussed or analyzed voluntourism indicated that there are many negative impacts of this tourism pattern because of many malpractices. Accordingly, these negative impacts were exacerbated, putting voluntourism activities in the crosshairs and criticisms. This study seeks to present a different perspective on dealing with voluntary tourism, not only as a tourism model, but also by presenting it as a business model. Applying Business Model Canvas for Voluntourism (BMC4V) opens up new strategies and policies for decision-makers to enhance this tourism pattern, reduce crises and the negative effects associated with it and optimise these activities. This leads to a reconsideration of voluntourism as one of the promising tourism patterns that have the potentials to revive the tourism industry, especially after the COVID-19 pandemic and the scrambling for discovering and promoting all available and possible tourism patterns. In this context, this study presented a comprehensive business model that presented a clear and precise picture of all the actors involved in the voluntourism industry, as well as presented suggested key guidelines for the most effective voluntourism actors in order to restructure, develop, and raising the efficiency of this tourism pattern from a business perspective.

Exploring Wellbeing and Remote Learning Using the Delphi Method: Engaging Teacher Education Students as Co-Producers of Practice

In UK Higher Education (HE), there is a growing awareness that wellbeing is central to student experience. Increasingly, HE agendas are promoting a targeted holistic institutional approach to wellbeing that supports students thrive in and beyond the student journey. This study positions students as experts of the student experience. Through student voice, the study explores teacher education students' experiences as a product of the synergistic effects of wellbeing and eco-systemic factors related to remote learning. In turn, findings feed into programme and practice developments that support positive student wellbeing. A conventional Delphi Method was used for its effectiveness addressing four different conditions: (1) accessing geographically dispersed populations, (2) overcoming unequal power dynamics, (3) supporting structured communication between experts on a topic leading to consensus building and decision making, and (4) engaging students as co-producers of practice to support positive wellbeing. Our findings provide new insight into the multiplicity of factors that interact with student wellbeing to benefit, challenge, or threaten student experience and the coping resources teacher education students rely on to maintain their education trajectories. These insights provide valuable understanding informing future teacher education programming and practice.

THE DELPHI TECHNIQUE IN DIGITAL TRANSFORMATION TOWARD INNOVATIVE LEARNING READINESS

COVID-19 turned many organizations' long-term wish lists into essential needs overnight. Therefore, innovative learning factors became an integral part of digital transformation that includes learning, learning to learn, which helps students develop a growth mindset belief system about their abilities. Therefore, the successful application of innovative learning depends on the students' readiness toward learning in a new norm. The aim is to collect expert-based judgments and often use them to identify consensus. The application of Delphi techniques in the digital transformation is to discuss the processes of the results. The findings of this research are summarized using systematic reviews of Delphi techniques and examined from a methodological perspective. Five systematic reviews of Delphi techniques from higher learning institutions were conducted and thoroughly analysed. Data were gathered from five experts. They are all associated with the sphere of education, with professors in statistical and research design relating to digital and innovative services, as well as academic associated fields. Much research on innovative or digital transformations have been published, but this study aims to develop consensus by using a series of questionnaires distributed in multiple iterations to collect data from a panel of selected experts. The Delphi results have confirmed indicators that can be successfully used as a benchmarking for digital transformation integration toward innovative education. Hence, several revisions to the Delphi technique have now been established. This method for evaluating instruments relies on human judgment to provide proof of the reliability of the content being evaluated. Ultimately, after three rounds of Delphi methodologies, the consensus was reached. Apart from that, it is also necessary to explain what specific criteria are used to evaluate the quality of their implementation and reporting. This model is believed to be a future reference for institutions of higher learning education to help understand and appreciate the design thinking approach and the ICT aspects when assessing the readiness for innovative learning. Based on the results, it is obvious that further research is needed to shed light on the methodological approaches and modifications of Delphi techniques. © 2022 Science Press. All rights reserved.

Underrepresentation of Respiratory Therapists as Experts in Delphi Studies on Respiratory Practices and Research Priorities.

Delphi survey techniques are a common consensus method used to collect feedback from an expert panel to inform practices, establish guidelines, and identify research priorities. Collecting respiratory therapists' (RT) expertise and experiences as part of consensus-building methodologies is one way to ensure that they align with RT practices and to better influence respiratory care practice. This narrative review aimed to report the RT representation in expert panels of Delphi studies focused on respiratory therapy practices and research priorities. The research question that guided this review is: to what extent are RTs included as expert participants among published Delphi studies relate to respiratory therapy and research topics? We conducted a structured search of the literature and identified 23 papers that reported Delphi studies related to respiratory care practices and 15 that reported on respiratory-related research priorities. Delphi studies that focused on reporting consensus on respiratory care practices included the following: (1) mechanical ventilation, (2) high-flow nasal cannula therapy, (3) COVID-19 respiratory management, (4) home oxygen therapy, (5) cardiopulmonary monitoring, and (6) disease-specific guidelines. Delphi studies that focused on establishing respiratory research priorities included the following: (1) theory and practice-orientated knowledge gaps, and (2) priority research topics for empirical investigation. The results of this review suggest that RTs were rarely included as expert participants and, when involved, were minimally represented (5% to 33%). Given RTs' diverse and relevant experience in respiratory care, incorporating their perspectives to inform future education, respiratory care practices, and research priorities would allow evidence to better align with knowledge gaps deemed important for the respiratory therapy profession.

Malaysian dental deans’ consensus on impact of COVID-19 and recommendations for sustaining quality dental education

The COVID-19 pandemic has significantly affected the training of dental students and the impact can be felt not just by the students, but patients seen at dental school operated clinics. We used the modified-Delphi method to investigate the response from deans of all Malaysian dental schools on the impact of COVID-19, and to solicit their views on policy recommendations to sustain quality dental education during and beyond the infectious disease outbreak. Our analysis revealed that all dental deans are in agreement with strong consensus to 10 out of fifteen items listed to be the challenges they faced due to the COVID-19 pandemic particularly these three items: “Patients under care of students experienced interrupted dental treatment which increases their risk of having their dental and overall wellbeing affected”;“Increased clinical budget for daily operations of Students’ clinics to cater for extra PPE and related expenses,” and;“The Students’ ability to meet clinical requirements for safe and competent practice.” All deans agreed with a strong consensus to the policy statements formulated based on the WHO’s Six Building Blocks to sustain quality dental education. In view of the results, we further discussed the importance of acknowledging that university-led dental clinics as an important part of the national healthcare system. Hence, we proposed that dental education need to have clear policies on having adequate number of trained staff, sufficient funds, information, supplies, transport, communications and overall guidance, and direction to function. Having a National Dental Education Policy that addresses the unique challenges identified in this study will serve as a monumental cornerstone to sustain quality dental education during times of calamity. Copyright © 2022 Mohd-Dom, Lim, Rani and Yew.

Expert Consensus on Measures to Promote Physical and Psychological Health among COVID-19-Related Healthcare Workers in Korea using Delphi Technique.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic has caused health problems and distress among healthcare workers (HCWs), so supportive measures to promote their health and relieve distress are needed. MATERIALS AND METHODS: We conducted two rounds of Delphi surveys with 20 COVID-19-related frontline healthcare professionals and public officials. The surveys evaluated means of supporting HCWs' health by improving health care systems and working environments in terms of effectiveness and urgency. The validity of the measures was assessed by calculating the content validity ratio. RESULTS: The top-priority measures to support HCWs were "secure isolation units capable of treating severe cases" in the facility infrastructure category, "secure nursing staff dedicated for patients in the intensive care units" in the personnel infrastructure category, "improve communication between central office and frontline field" in the cooperation system category, "support personal protective equipment and infection control supplies" in the aid supplies category, and "realization of hazard pay" in the physical/mental health and compensation category. CONCLUSION: There was consensus among the experts on the validity and priorities of policies in the facility, personnel, cooperation, supplies, and compensation categories regarding measures to promote COVID-19 related HCWs' health.

Consensus recommendations for opioid agonist treatment following the introduction of emergency clinical guidelines in Ireland during the COVID-19 pandemic: A national Delphi study.

BACKGROUND: Emergency contingency guidelines for opioid agonist treatment (OAT) were introduced in Ireland in March 2020, to ensure rapid and uninterrupted access to treatment while mitigating COVID-19 risk. The contingency guidelines deviated, across multiple clinical domains, from pre-pandemic clinical guidelines published in 2016. The objectives of this study are to (1) identify changes introduced to OAT clinical guidelines in Ireland during the pandemic; and (2) develop consensus on whether the new recommendations should be retained beyond the pandemic, using a national Delphi consensus methodology. METHODS: Clinical guidance recommendations ('statements') were generated by comparing the newly established contingency guidelines with the national 2016 Clinical Guidelines for OAT. Over two rounds of on-line Delphi testing, a panel of experts (people currently accessing OAT, psychiatrists, general practitioners, community pharmacists, a nurse, a psychologist and support/key workers) independently rated their agreement with each statement and provided comments. Statements with a median score of 4 or 5 and a lower quartile of ≥4 were classified as having reached consensus. RESULTS: Forty-eight panel members were recruited, with a high participation level at Round 2 (90%, n=43). Consensus was achieved for 12 of the 19 statements at Round 1. The 7 remaining statements were revised, with 2 new statements, resulting in 9 statements at Round 2. Four statements reached consensus at Round 2. The final list includes 16 clinical guidance statements; 9 relating to assessment, 3 to OAT drug choice and dosing, 1 to take-away doses, 2 to overdose prevention and 1 to the continuation of e-prescriptions. CONCLUSIONS: A wide range of stakeholders involved in the delivery and receipt of OAT agreed on 16 clinical guidance statements for inclusion in OAT clinical guidelines as we move beyond the pandemic, rather than reverting to pre-pandemic guidelines. The agreed statements relate to facilitating safe access to OAT with minimal waiting time, supporting patient-centred care to promote health and well-being, and preventing drug overdose. Notably, consensus was not achieved for OAT drug dosage and frequency of urine testing during the stabilisation and maintenance phase of care.

The Investigation of Key Factors in Polypropylene Extrusion Molding Production Quality

This study took food-grade polypropylene packaging products as the research project and discussed how to control the polypropylene extrusion sheet thickness and vacuum thermoforming quality and weight. The research objective was to find the key factors for reducing costs and energy consumption. The key aspects that may influence the polypropylene extrusion molding quality control were analyzed using literature and in-depth interviews with scholars and experts. These four main aspects are (1) key factors of polypropylene extrusion sheet production, (2) key factors of the extrusion line design, (3) key factors of polypropylene forming and mold manufacturing, and (4) key factors of mold and thermoforming line equipment design. These were revised and complemented by the scholar and expert group. There are 49 subitems for discussion. Thirteen scholars and experts were invited to use qualitative and quantitative research methods. A Delphi questionnaire survey team was organized to perform three Delphi questionnaire interviews. The statistical analyses of encoded data such as the mean (M), mode (Mo), and standard deviation (SD) of various survey options were calculated. Seeking a more cautious research theory and result, the K-S simple sample test was used to review the fitness and consistency of the scholars’ and experts’ opinions on key subitem factors. There are ten key factors in the production quality, including “A. Main screw pressure”, “B. Polymer temperature”, “C. T-die lips adjustment thickness”, “D. Cooling rolls pressing stability”, “E. Cooling rolls temperature stability”, “F. Extruder main screw geometric design”, “G. Heating controller is stable”, “H. Thermostatic control”, “I. Vacuum pressure”, and “J. Mold forming area design”. The key factors are not just applicable to classical polypropylene extrusion sheet and thermoforming production but also to related process of extrusion and thermoforming techniques in expanded polypropylene (EPP) sheets and polylactic acid (PLA). This study aims to provide a key technical reference for enterprises to improve quality to enhance the competitiveness of products, reduce production costs, and achieve sustainable development, energy savings, and carbon reductions.

Using maternal and neonatal data collection systems for coronavirus disease 2019 (COVID-19) vaccines active safety surveillance in low- and middle-income countries: an international modified Delphi study: [version 1;peer review: 1 approved with reservations]

Background: Given that pregnant women are now included among those for receipt coronavirus disease 2019 (COVID-19) vaccines, it is important to ensure that information systems can be used (or available) for active safety surveillance, especially in low- and middle-income countries (LMICs). The aim of this study was to build consensus about the use of existing maternal and neonatal data collection systems in LMICs for COVID-19 vaccines active safety surveillance, a basic set of variables, and the suitability and feasibility of including pregnant women and LMIC research networks in COVID-19 vaccines pre-licensure activities. Methods: A three-stage modified Delphi study was conducted over three months in 2020. An international multidisciplinary panel of 16 experts participated. Ratings distributions and consensus were assessed, and ratings’ rationale was analyzed. Results: The panel recommended using maternal and neonatal data collection systems for active safety surveillance in LMICs (median 9;disagreement index [DI] -0.92), but there was no consensus (median 6;DI 1.79) on the feasibility of adapting these systems. A basic set of 14 maternal, neonatal, and vaccination-related variables. Out of 16 experts, 11 supported a basic set of 14 maternal, neonatal, and vaccination-related variables for active safety surveillance. Seven experts agreed on a broader set of 26 variables.The inclusion of pregnant women for COVID-19 vaccines research (median 8;DI -0.61) was found appropriate, although there was uncertainty on its feasibility in terms of decision-makers’ acceptability (median 7;DI 10.00) and regulatory requirements (median 6;DI 0.51). There was no consensus (median 6;DI 2.35) on the feasibility of including research networks in LMICs for conducting clinical trials amongst pregnant women. Conclusions: Although there was some uncertainty regarding feasibility, experts recommended using maternal and neonatal data collection systems and agreed on a common set of variables for COVID-19 vaccines active safety surveillance in LMICs.

Modelli regionali di partecipazione in sanità. Uno studio comparativo

One of the main dynamics of change in the Italian welfare system since the 1970s has been the restructuring of decision-making processes to encourage participation. Today, the new millennium presents evidence of the growing protagonism of the public service authorities in this process. This leading role heralds ambivalent outcomes;although it expresses an attempt to retain the public service’s role as guarantor of fundamental rights, it also lends itself to processes of de-accountability of the same public body. Twenty years after the reform of Title V of the Italian Constitution, this article examines and compares the institutional representations of the role of citizens-users/patients in five regional health systems in Italy. Our study, conducted using the Delphi technique, identified the instruments currently applied and the impact of the 2020/2021 Covid pandemic on the regional systems. The results highlight some interesting experiments in the direction of overcoming the traditional “democratic deficit” of the health system;even if the organizational, communicative, cultural and training challenges of “competent” citizens and communities remain open and further accentuated after the Covid-19 pandemic. Copyright © FrancoAngeli

Prioritizing Global Public Health Investments for COVID-19 Response in Real Time: Results from a Delphi Exercise.

In the first months of the COVID-19 pandemic, there was a lack of guidance on how to channel the unprecedented amount of health financing toward the pandemic response. We employed a multistep, interactive Delphi process to reach consensus on a "menu" of priority COVID-19 response interventions. In all, 27 health security experts-representing national governments, bilateral and multilateral organizations, academia, technical agencies, and nongovernmental organizations-participated in the exercise. The experts rated 11 technical investment areas and 37 interventions on a 5-point scale in terms of their importance to COVID-19 response. Initial findings were discussed at a virtual meeting where experts suggested modifications. A group of 19 experts then rated a revised list of 11 technical areas and 39 interventions. Consensus was defined as at least 80% of experts agreeing on the importance of a technical area or intervention; stability of scores across the rounds was identified using Wilcoxon matched-pairs and unpaired signed rank tests. Between the initial and final menu, 3 technical areas and 7 interventions were slightly modified, 3 interventions were added, and 1 intervention was removed. Consensus was reached on all 11 technical areas and 35 of the final 39 interventions, and between 34 and 37 interventions were stable across rounds depending on the test used. In this exercise, the health security experts agreed that COVID-19 response financing should prioritize interventions that enhance a country's capacity to test, trace, and treat high-risk populations. Simultaneously, supportive systems (eg, risk communication, community engagement, public health infrastructure, information systems, policy and coordination, workforce capacity, other social protections) should be developed to ensure that nonpharmaceutical and medical interventions can maximize the effectiveness of these systems.

Developing a National Set of Health Equity Indicators Using a Consensus Building Process.

BACKGROUND: Considerable health inequities documented in Israel between communities, populations and regions, undermine the rights of all citizens to optimal health. The first step towards health equity is agreement on a set of national indicators, reflecting equity in healthcare provision and health outcomes, and allowing monitoring of the impact of interventions on the reduction of disparities. We describe the process of reaching a consensus on a defined set of national equity indicators. METHODS: The study was conducted between January 2019 and June 2020, in a multistage design: (A) Identifying appropriate and available inequity measures via interviews with stakeholders. (B) Agreement on the screening criteria (public health importance; gap characteristics; potential for change; public interest) and relative weighting. (C) Constructing the consultation framework as an online, 3-round Delphi technique, with a range of experts recruited from the health, welfare and education sectors. RESULTS: Participants were of diverse age, gender, geographic location, religion and ethnicity, and came from academia, healthcare provision, government ministries and patient representative groups. Thirty measures of inequity, presented to participants, represented the following domains: Health promotion (11 indicators), acute and chronic morbidity (11), life expectancy and mortality (2), health infrastructures and affordability of care (4), education and employment (2). Of the 77 individuals contacted, 75 (97%) expressed willingness to participate, and 55 (73%) completed all three scoring rounds. The leading ten indicators were: Diabetes care, childhood obesity, adult obesity, distribution of healthcare personnel, fatal childhood injuries, cigarette smoking, infant mortality, ability to afford care, access to psychotherapy and distribution of hospital beds. Agreement among raters, measured as intra-class correlation coefficient (ICC), was 0.75. CONCLUSION: A diverse range of consultants reached a consensus on the most important national equity indicators, including both clinical and system indicators. Results should be used to guide governmental decision-making and inter-sectoral strategies, furthering the pursuit of a more equitable healthcare system.

New implementations for the ASPEN clinical guidelines and a call for protocol review on the nutrition guidelines for adult head and neck cancer.

The current article details new upgrades to the guideline methodology, including the publication of "a priori" protocols, methods for assuring agreement between reviewers, adoption of Covidence software for tracking reasons for article exclusion, the establishment of a separate bias panel to improve study quality assessment, an algorithm for study design inclusion, and Delphi technique implementation for the management and external validation of expert opinion. These methodologic additions will provide greater transparency and accountability for the choices made during the guideline development process.

Delphi methodology in healthcare research: How to decide its appropriateness.

The Delphi technique is a systematic process of forecasting using the collective opinion of panel members. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine. The Delphi methods assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically/logistically difficult or evidence is conflicting. However, the attempts to assess the quality standard of Delphi studies have reported significant variance, and details of the process followed are usually unclear. We recommend systematic quality tools for evaluation of Delphi methodology; identification of problem area of research, selection of panel, anonymity of panelists, controlled feedback, iterative Delphi rounds, consensus criteria, analysis of consensus, closing criteria, and stability of the results. Based on these nine qualitative evaluation points, we assessed the quality of Delphi studies in the medical field related to coronavirus disease 2019. There was inconsistency in reporting vital elements of Delphi methods such as identification of panel members, defining consensus, closing criteria for rounds, and presenting the results. We propose our evaluation points for researchers, medical journal editorial boards, and reviewers to evaluate the quality of the Delphi methods in healthcare research.